Benjamin Knorlein is a Computer Scientist from the Center for Computation and Visualization from Brown University, he is working on the design of software that will enable scientists to study plankton through virtual reality.
Every great ship needs a Holodeck. Not for entertainment, but for science. Star Trek‘s fictional Holodeck can create matter virtually (in our case, it would be scientific data) which can be seen, touched and interacted with. Unfortunately, real technology is not as advanced as we know it from TV, so for now we skip the touch. But there is already a lot we can do within Virtual Reality to immerse ourselves in the data we collect by seeing and interacting with it in a virtual world.
What do we need to get the Falkor ready for exploration?
- Get a commercially available Virtual Reality system and install it in the science control room – Check.
- Acquire data. Lots of data. And then even more data. – Check. Check. Check.
- Step into the virtual wonderland of science – Beam me up, Scotty!
Become a Phytoplankton
On board we have holographic microscope. (Yes, holographic!) In contrast to a normal microscope, the recorded holograms can refocus the microscopic image at different distances to the camera. So instead of a normal image, we acquire three-dimensional information about the location of phytoplankton in the water column. By using the the high-performance computers onboard Falkor, combined with algorithms I wrote, we can identify the kind of phytoplankton we are looking at and know its size and its location. When we plug the data in Virtual Reality, we can wander in the phytoplankton universe and see how the little organism organize themselves in the water. Scientists – who usually stare at a screen looking at phytoplankton for hours – can suddenly become part of the world they study so hard, thereby developing new perspectives, including the perspective of a phytoplankton.
This is a holographic picture of a Diatom Chain as it looks when it is captured by the camera that descends together with the CTD Rosette.
This image merges all of the plankton organisms found in a drop of water the size of a needle’s head. The image is now ready to be processed to become part of a virtual reality environment.
Take a Hike on the Seafloor or Enter the Twilight Zone
The R/V Falkor has two high resolution multibeam echo sounders. The sonars use sound to identify seafloor features in 3D. By plugging the data into Virtual Reality we can walk on the seafloor and become geologists of the ocean.
A lot of our equipment on board samples and records data in the twilight zone of the ocean – the region below the sunlit surface. The ‘wire walker’ is an instrument currently being used by the experts here – it moves up and down on a wire 150 times a day, measuring data such as temperature, salinity, pressure, oxygen, fluorescence of the phytoplankton, sunlight, and turbidity. Quite a busy explorer, this wirewalker. Taking this data combined with its GPS reporting and plugged into the Virtual Reality we can explore the depths of the ocean where no one has gone before.
Virtual Reality and New Scientific Possibilities
As the Sea to Space expedition progresses, on board personnel on board enjoy immersing themselves in the plankton communities collected through holographic cameras through virtual reality.
As you can see, Virtual Reality offers new possibilities to interact with scientific data. The experience is often more personal and helps to develop a different relation with the data as we coexist with it in the same world. We are suddenly not separated from it by a computer screen, but can enter its universe. We can shrink to the size of phytoplankton or we can walk deep in the oceans. We can create and modify matter and can interact with the target of our research not by using a mouse or keyboard, but by using natural interactions, e.g. we simply move our head or walk around to see the data from different perspectives.
We might also play a round of minigolf or draw phytoplankton in 3D. All, of course, for pure science.
Tags: Sea to Space 2017